Searching for visual content has become more common as digital images, video recordings, and the like have become ever more widespread due, at least in part, to the propagation of numerous types of inexpensive digital imaging and recording devices, and the extensive availability of the resulting visual content on the Internet. Further, for example, the growth and accessibility of community-contributed media content on the Internet has led to a surge in the use of visual searching tools for locating desired image or video content. However, due to the great success of text-based searching tools, most popular image and video search engines, such as those provided by Google®, Yahoo!®, and Bing™, are built on text-based searching techniques, such as by relying on text associated with visual content for returning results in response to a visual search query. This approach to searching for visual content has proven unsatisfying, as it often entirely ignores the visual content itself as a ranking indicator in determining the most relevant results.
To address this problem, the subject of visual search result reranking has received increasing attention in recent years. Search reranking can be defined as the reordering of the located visual documents based on multimodal cues to improve the relevancy of the search results. For example, the results being reordered might be image files, video recordings, keyframes, or the like returned in response to a search query in an initial ranked order. Conventional research on visual search reranking has tended to proceed along two main directions: (1) self-reranking which only uses the initial search results for reranking of the results; and (2) query-example-based reranking which leverages user-provided query examples and results for reranking of the results.